Abstract

Since 1993, the Cochrane Collaboration has been pursuing the vision that health care decision making around the world should be informed by high-quality and timely research evidence. The Collaboration follows several key principles, one of which is “striving for relevance.” To achieve this, the general structure of the Collaboration has been set up to facilitate the involvement of clinicians and patients in developing high-quality evidence through a rigorous process. To increase the impact of its reviews, the Collaboration has over the last 5 years recognized the need for a more accountable and systematic approach to selecting research questions for systematic reviews. The recognition of the importance of identifying the right questions has also led to the development of another initiative in the United Kingdom with which the Collaboration has an increasing working relationship, the James Lind Alliance. The James Lind Alliance facilitates the creation and work of Priority Setting Partnerships to identify uncertainties on specific health problems. Several Cochrane review groups work with the James Lind Alliance to identify and prioritize uncertainties patients and clinicians have about the effects of treatments. However, as the James Lind Alliance is a UK-based initiative, this introduces a challenge for the Collaboration to look at the broader picture and identify different approaches to ensure an international aspect in selecting topics for Cochrane reviews
Hide

Abstract

This report is about setting priorities for health technology assessment (HTA). HTA examines systematically the consequences of the application of health technologies (broadly defined to include any health care intervention) to support decision making in policy and practice. Only a fraction of existing health technologies have been formally evaluated, and many more appear each year. Resources for HTA are, however, limited so that priorities have to be set, whether explicitly or implicitly. The aim of setting priorities for HTA should be to identify those assessments that offer the greatest benefits in relation to their cost, and thus to maximize the benefit derived from investments in HTA.
Hide

Abstract

OBJECTIVES: We aimed to develop and pilot a process for joint working between Cochrane Review Groups (specialist-area groups responsible for producing Cochrane reviews) and Cochrane Fields (broad-spectrum interest groups), for identifying high priority review topics and enhancing quality and dissemination of priority reviews. STUDY DESIGN AND SETTING: We developed and piloted a framework for collaboration between a Cochrane Review Group (specializing in musculoskeletal injuries) and a Cochrane Field (focusing on health care of older people) for identifying, delivering, and disseminating priority Cochrane intervention reviews using hip fracture rehabilitation as an exemplar. The processes adopted included consultation of members of both the entities, mapping of trials from the Review Group's Specialized Register, jointly establishing criteria for topic prioritization, identification of researchers, and facilitating provision of expert peer review from the field. RESULTS: A framework for effective collaboration between a Cochrane Review Group and Cochrane Field for identifying and delivering priority Cochrane Reviews was devised and piloted. Additionally, two new Cochrane reviews, preceded by protocols, were published. CONCLUSION: The project demonstrated the feasibility and potential benefits of a structured collaboration between a Cochrane Review Group and a Cochrane Field for the identification and production of Cochrane reviews on priority topics.
Hide

Abstract

Objectives: To promote the further development of HTA, this study aims to design a model for HTA priority setting, which would address national needs for a country with little experience in the field of HTA, and assess its feasibility for the health system. Methods: Literature search and review, as well as qualitative research have been used in the design and testing of the model for HTA priority setting. To test the model and the methodology, a three-round Delphi study was conducted in 2011 in the form of an electronic questionnaire, which was distributed to the panel of eleven national experts. The panel was composed of experts representing various fields of the health care sector: policy-makers, health care service professionals and academics, with diverse professional roles. Results: The designed model consists of four stages: (i) selection of experts for the panel, (ii) indication and selection of health policy topics, (iii) identification of health technologies, and (iv) priority setting. Three rounds of the Delphi study were performed to test the model and reach expert consensus on a list of health technologies for assessment, including pharmaceuticals, vaccines, medical devices, diagnostic methods, public health interventions, organizational systems, etc. Conclusions: Based on the Delphi technique as a method for consensus building, the model for HTA priority setting was developed for Lithuania; however, it could also be used for other countries with little experience in the field of HTA.
Hide

Abstract

OBJECTIVES: This study aimed to develop and pilot an equity lens that could help researchers in developing a more equity-oriented approach toward priority setting and agenda setting in systematic reviews. STUDY DESIGN AND SETTING: We developed an equity lens to guide the development and evaluation of a prioritization process and evaluate its outcomes based on the information derived from a discussion workshop and a comparison with the existing literature on the topic. We piloted the process section of the equity lens across the 13 structured priority-setting approaches in the Cochrane Collaboration. RESULTS: We devised an equity lens with two checklists: one to guide the process of priority setting (nine questions) and the other to evaluate the outcomes of priority setting (eight questions). Of the nine questions, seven questions were partially addressed by at least one of the prioritization projects. Two questions were not considered in any of them. The prioritization projects did not report sufficient outcome data, thus we could not explore the eight question on evaluating outcomes. CONCLUSION: Currently, there are few strategies in the Cochrane Collaboration that explicitly address the research priorities of individuals from different sociodemographic groups. The equity lens for priority setting and agenda setting can help project teams to develop a more equity-oriented approach to set a research agenda and/or prioritize research topics. However, further studies are needed to evaluate its impact on the prioritization process. Copyright © 2013 Elsevier Inc. All rights reserved.
Hide

Abstract

Aim: With rapid innovations in diagnostic and therapeutic interventions in cancer care, comparative effectiveness reviews (CERs) are essential to inform clinical practice and guide future research. However, the optimal means to identify priority CER topics are uninvestigated. We aimed to devise a transparent and reproducible process to identify ten to 12 CER topics in the area of cancer imaging relevant to a wide range of stakeholders. Materials & methods: Environmental scans and explicit prioritization criteria supported interactions (email communications, web-based discussions and live teleconferences) with experts and stakeholders culminating in a three-phase deductive exercise for prioritization of CER topics. Results: We prioritized 12 CER topics in breast, lung and gastrointestinal cancers that addressed screening, diagnosis, staging, monitoring and evaluating response to treatment. Conclusion: Our project developed and implemented a transparent and reproducible process for research prioritization and topic nomination that can be further refined to improve the relevance of future CERs.
Hide

Abstract

Summary points There is no consensus on appropriate methods for deciding when to update systematic reviews A decision tool was developed to replace an approach based on an arbitrary and rigid time period with a priority based approach The tool broadly consists of three criteria: clinical question answered or no longer relevant, new relevant factors to consider, and availability of new studies The decision tool can help identify reviews most sensitive to change and thus minimise unnecessary updating and waste of resources
Hide

Abstract

Development and assessment of methods for setting priorities for systematic reviews is developing into a field of increasing interest amongst such groups as the AHRQ and NICE [1] and [2]. The Cochrane Collaboration has recently established a new Methods group entitled the “Cochrane Agenda and Priority Setting Methods Group” to develop methodology and advise The Cochrane Collaboration on how the validity and precision of systematic reviews can be improved. Over the last several years, the Cochrane Collaboration has recognized the need for a more accountable and systematic approach to selecting research questions for systematic reviews. When the Cochrane Collaboration was established 20 years ago (the 20th year celebration will take place at the Quebec City Cochrane Colloquium in September of this year) the topic selection was 'expert/investigator-driven,'in that the authors conducted systematic reviews on topics they were interested in. The enthusiasm of these expert authors was one reason the Cochrane Collaboration has been so successful, with the over 4000 systematic reviews in the Cochrane Library. However, a more systematic approach is needed to address gaps of importance to patients, practitioners, and policymakers. The Collaboration recognized this and in 2007, they established a one-off initiative entitled the “Prioritization Fund,” which would suggest mechanisms for improving the relevance of Cochrane Reviews. In this month's issue of the Journal of Clinical Epidemiology, we have some of the products of this initiative. Following two commentaries commentary 1 commentary 2 on the importance of prioritizing systematic review questions, there are six reviews and original articles that report the results of several of the Prioritization Fund projects.
Hide

Abstract

Background: The Cochrane Eyes and Vision Group (CEVG) US Project was funded by the Cochrane Collaboration Prioritization Fund from 2007 to 2009 to test a framework for prioritizing systematic reviews. Objectives: To describe the impact, accountability, and sustainability of the project. Methods: A summary of research activities, professional collaborations, funding, and research opportunities. Results: We identified priority review topics related to open angle glaucoma using practice guidelines and a Delphi survey of clinicians. We searched for and appraised the methodological quality of existing systematic reviews for priority topics (Cochrane and non-Cochrane). For topics without any high quality review, we sought approval from the CEVG editor and posted the topics on the CEVG website. For priority topics with non-Cochrane reviews, we encouraged authors to convert them into Cochrane reviews as appropriate. We presented project findings at each stage, at the Cochrane Colloquia (2008 to 2010) and other international meetings. This project comprised one part of TL's PhD thesis, and has resulted in two publications. An additional manuscript is under revision. We involved various stakeholders in the project, including CEVG and its Advisory Board, the US Cochrane Center, local biostatisticians and epidemiologists, guideline developers at the American Academy of Ophthalmology, and clinicians and vision research scientists from the American Glaucoma Society and Asian-Pacific region. Findings from this project also provided the groundwork for an additional grant award to our team from the National Eye Institute, to develop research priorities related to diabetic retinopathy and primary angle-closure glaucoma. Two Master's students are pursing the newly-funded projects as their thesis research. The American Optometric Association is considering collaboration with us to update their guidelines on diabetic eye disease. Conclusions: The impact of the Collaboration's Prioritization initiative has been profound for our group and our research area. We have demonstrated accountability to the Collaboration and sustainability of the framework.
Hide

Abstract

BACKGROUND: Currently there is no framework for those involved in the identification, evaluation and prioritisation of new diagnostic technologies. Therefore we aimed to develop prioritisation criteria for the assessment of new diagnostic technologies, by gaining international consensus on not only which criteria should be used, but also their relative importance. METHODS: A two-round Delphi process was used to generate consensus amongst an international panel of twenty-six experts on priority criteria for diagnostic health technology assessment. Participants represented a range of health care and related professions, including government, industry, health services and academia. RESULTS: Based on the responses to the first questionnaire 18 criteria were placed into three categories: high, intermediate and moderate priority. For 16 of the 18 criteria, agreement with the categorisation of the criteria into the high, intermediate and moderate categories was high at > or = 70% (10 had agreement > or = 80%). A further questionnaire and panel discussion reduced the criteria to 16 and two categories; seven were classified as high priority and nine intermediate. CONCLUSIONS: This study proposes an objective structure of prioritisation criteria to use when assessing new diagnostic technologies, based on an expert consensus process. The value of these criteria is that no one single component should be used as the decisive driver for prioritisation of new diagnostic technologies for adoption in healthcare settings. Future studies should be directed at establishing the value of these prioritisation criteria across a range of healthcare settings.
Hide

Abstract

Objectives To develop a method of economic evaluation and triage for research prioritisation, before the funding decision. Data sources Existing models were researched focusing on MEDLINE, HealthSTAR, IBSS and HEED. Review methods Papers of primary relevance that included a proposed model were reviewed in detail, and their models appraised using criteria adapted from the EUR-ASSESS project and the authors'previous experience. From this the PATHS model was developed. It assumes three or more possible alternative outcomes or scenarios in terms of research results: 'favourable'to the technology being assessed, 'unfavourable'or 'inconclusive'. An associated flow of benefits or disbenefits, costs or savings is identified for each potential research outcome depending on the likely implementation of the results as judged by experts. These benefits and costs are weighted and discounted in the model to give an expected incremental cost-effectiveness ratio (EICER). EICERS could be estimated for any number of research areas or proposals to inform funding prioritisation. The model was tested and evaluated on three case studies identified in liaison with the NHS R&D HTA programme and the UK Medical Research Council. These case studies were funded research projects, where full evaluation was underway and where results would be reported during the PATHS project. The studies were selected to include surgery or other invasive procedures, and non-invasive health services projects (a fourth case study did not complete during the course of the study). The three case studies included randomised controlled trials of early surgery or observation for small abdominal aortic aneurysms, infusion protocols for adult pre-hospital care, and postnatal midwifery support. Results Each of the three assessments indicated net clinical benefit or no clinical loss of benefit, in addition to health service cost savings in excess of the cost of the trial. For two case studies, the value of the proposed trial, as evaluated by the model in the prediction, was consistent with the ex post evaluation, thus providing positive tests of the value of the model. In the third case meaningful ex post analysis was not possible as very poor compliance with the trial protocol (indicated in the ex ante evaluation) seriously undermined its conclusions. During the study, at the request of the UK HTA programme, the model was also applied to a funding request for a large randomised trial of beta-interferon for multiple sclerosis treatment. Conclusion The PATHS model has a useful part to play in the research prioritisation process. Its strengths lie in its emphasis on the impact of research results on policy and practice (the keystone for NHS research) and net effects on health benefits and costs. It assesses the cost-effectiveness of the research and may identify ways to enhance the research design, endpoints relevant to implementation, analytical methods and dissemination. Further research is recommended to investigate the scope for synthesising the strengths of the PATHS model with other approaches including value of information; to compare ex ante and immediate ex post assessments of implementation with long term follow-up of actual implementation; and to assess the robustness of such approaches to the choice and number of experts used.
Hide

Abstract

BACKGROUND: Decision makers in health care organizations struggle with how to set priorities for new technologies in medicine. Traditional approaches to priority setting for new technologies in medicine are insufficient and there is no widely accepted model that can guide decision makers. DISCUSSION: Daniels and Sabin have developed an ethically based account about how priority setting decisions should be made. We have developed an empirically based account of how priority setting decisions are made. In this paper, we integrate these two accounts into a transdisciplinary model of priority setting for new technologies in medicine that is both ethically and empirically based. SUMMARY: We have developed a transdisciplinary model of priority setting that provides guidance to decision makers that they can operationalize to help address priority setting problems in their institution.
Hide

Abstract

This article describes the actual situation at the beginning of 1999 with regard to identification and priority setting for health technology assessment (HTA) on a national level in the Netherlands. For this purpose the literature on HTA published in 1980-1998, mainly national, was thoroughly reviewed. Many policy documents and other reports from the 'grey literature'of identification and priority setting for HTA in the Netherlands were also used. The results show that attempts to identify and set priorities for HTA is a new activity in the Netherlands. The three most important actors in the field are the Health Council, the Council for Health Research and the Health Insurance Council. Methodologies differ depending on the content and scope of each programme. In addition, the methods used are not always transparent and the activities are not co-ordinated. The lack of co-ordination is due to the fact that there is no single organisation that is authorized to identify and set priorities for HTA. Suggestions for improving co-ordination are proposed with the aim of developing a truly national effort in this field, which will enable a more balanced and efficient set of HTA activities.
Hide

Abstract

OBJECTIVES: Health policy making is complex, but can be informed by evidence of what works, including systematic reviews. We aimed to inform the work of the Cochrane Effective Practice and Organisation of Care (EPOC) Group by identifying systematic review topics relevant to Australian health policy makers and exploring whether existing Cochrane reviews address these topics. METHODS: We interviewed 30 senior policy makers from State and Territory Government Departments of Health to identify topics considered important for systematic reviews within the scope of health services research, including professional, financial, organisational and regulatory interventions to improve professional practice and the organisation of services. We then looked for existing Cochrane reviews relevant to these topics. RESULTS: Eighty-five priority topics were identified by policy makers, including advanced practice roles, care for Indigenous Australians, care for chronic disease, coordinating across jurisdictions, admission avoidance, and eHealth. Sixty published Cochrane reviews address these issues, and 34 additional reviews are in progress. Thirty-four topics are yet to be addressed. CONCLUSIONS: This survey has identified questions for which Australian policy makers have indicated a need for systematic reviews. Further, it has confirmed that existing reviews do address issues of importance to policy makers, with the potential to inform policy processes.
Hide

Abstract

BACKGROUND: Systematic reviews are fundamental decision-making tools for the assessment of the effectiveness of healthcare interventions and for determining future research fields. In the field of hand surgery, so far, there exists only a limited number of systematic reviews. According to the diversity and multitude of hand surgical topics, relevant fields for systematic reviews were identified by means of an expert ranking. MATERIAL AND METHODS: A catalogue of 24 topics was extracted from publications of representative hand surgical journals. These topics were categorized and integrated into a questionnaire which was presented to 30 experts in the field of hand surgery. Every single topic had to be rated by marks from 1 (very relevant) to 5 (not relevant). Further topics could have been indicated. Averaging of the marks allowed to define a ranking and a prioritisation of the topics. RESULTS: 29 of 30 questionnaires were returned. The ranking showed scapholunate ligament injury, Kienboeck's disease and scaphoid nonunion to have the highest relevance, followed by 5 other topics dealing with wrist problems. The lowest relevance was attributed to neck fractures of the fifth metacarpal, ulnar collateral ligament injury of the thumb metacarpophalangeal joint and postoperative management following replantation. CONCLUSION: The results of the presented survey allow for a prioritisation of topics for systematic reviews in hand surgery. The ranking of the topics emphasises the particular relevance of injuries and diseases of the wrist.
Hide

Abstract

OBJECTIVES: Uncertainty is pervasive in decision making on new health technologies; therefore, some countries have put systems in place to support decision makers with timely information. An important, but as yet undocumented, determinant of the potential value for decision making of these so-called horizon scanning systems (HSSs) is how the most significant health technologies are selected. METHODS: All thirteen member organizations of EuroScan, a collaborative network for HSSs, were surveyed and interviewed on how they prioritize technologies for assessment. RESULTS: The majority of HSSs directly serves a customer. Some customers actively request early assessments of new health technologies, thereby diminishing the need for priority setting for the HSSs. All systems express a concern to miss an important technology and/or to select an unimportant technology. Almost all HSSs use explicit selection criteria, but these criteria hardly ever are operationalized. The number of criteria used varies, but costs and health benefit of the technology are always taken into account. The process of reaching a final decision is implicit, undocumented in all but one system, and is based on agreement by consensus. CONCLUSIONS: The process of making the final decision on which technologies to assess can be improved by applying existing criteria more consistently and transparently. Current practice does not safeguard against missing an important technology. This finding is probably most important to act upon for systems with customers that do not actively request assessment of specific technologies.
Hide